Allowlist registry redirect targets for private registries - #341

What are you trying to accomplish?

Self-hosted JFrog Artifactory instances redirect package downloads to presigned URLs on the customer's own S3 bucket (jfrog-prod-<id>.s3.amazonaws.com). The egress allowlist covers JFrog's shared regional buckets but not these per-instance ones, and the bucket name can't be derived from the registry host. So with egress enforcement on, downloads fail even though the index lookups work.

This PR adds EgressAllowlistHandler.HandleResponse. When one of the job's private registries returns a redirect, the proxy allows the redirect's exact HTTPS destination host for the rest of the job.

Anything you want to highlight for special attention from reviewers?

• Which redirects are trusted: only those from credential hosts that are not already on the static allowlist. Public registries and GitHub hosts are excluded because users can publish content there.
• What is learned: only the exact hostname of an https target. IP addresses are never learned, and sibling or parent hosts stay blocked.
• Alternatives considered: hard-coding each customer's bucket in the YAML doesn't scale. A jfrog-prod-*.s3.amazonaws.com wildcard is unsafe because anyone can register a bucket with a matching name. Following the actual redirect relies on a registry the job already trusts.
• Caching and concurrency: cached 3xx responses go through the same response hook, so they are handled too. The learned-host set is protected by an RWMutex.

How will you know you've accomplished your goal?

• New end-to-end test TestProxyEgressAllowlistEnforceAllowsPrivateRegistryRedirectTarget: the bucket returns 403 before the redirect. A download from the registry then follows the redirect and gets 200, with no credential sent to the bucket. Other hosts stay blocked.
• New unit tests: cover the rejected cases (redirects from hosts that aren't credential hosts, public or GitHub hosts, non-3xx responses, http targets, IP addresses, and a missing Location).
• Commands:
  • go test -race -count=1 ./...: passed.
  • golangci-lint: not run; it isn't installed locally.

Checklist

• I have run the complete test suite to ensure all tests and linters pass.
• I have thoroughly tested my code changes to ensure they work as expected, including adding additional tests for new functionality.
• I have written clear and descriptive commit messages.
• I have provided a detailed description of the changes in the pull request, including the problem it addresses, how it fixes the problem, and any relevant details about the implementation.
• I have ensured that the code is well-documented and easy to understand.